Skip to content

fix(dav): finalize upload metadata before post-write hooks#60453

Open
joshtrichards wants to merge 7 commits into
masterfrom
jtr/fix-dav-put-bookkeeping-consistency
Open

fix(dav): finalize upload metadata before post-write hooks#60453
joshtrichards wants to merge 7 commits into
masterfrom
jtr/fix-dav-put-bookkeeping-consistency

Conversation

@joshtrichards
Copy link
Copy Markdown
Member

@joshtrichards joshtrichards commented May 15, 2026

Summary

Tighten the DAV PUT finalization sequence in apps/dav/lib/Connector/Sabre/File.php so post-write hooks observe a more fully finalized file state, while preserving historical shared-lock behavior during hook execution.

What changed

  • Extract post-move finalization logic into finalizeUpload()
  • Keep exclusive lock through cache reconciliation, mtime/ctime/upload_time, checksum, and info refresh
  • Downgrade to shared lock before emitting post-write hooks
  • Consolidate checksum persistence into the single putFileInfo() call (previously a separate call after hooks)

Why

Previously the lock downgraded too early — before checksum and metadata were fully persisted — creating a window where the file was visible at the final path with incomplete bookkeeping state. This is a likely cause of intermittent FilesDrop CI failures:

{"message":"Cannot set extra headers for non-existing file 'files/jHybAbajiZcXgfy/Alice/folder (2)'"}
{"message":"Could not open file: drop/Alice/folder/a.txt (214), file does seem to exist"}

Cc: @icewind1991 — may be related to the logging work in #56166.

Locking sequence after this change

  1. Acquire shared lock before PUT
  2. Upgrade to exclusive for final publish step
  3. Finalize cache + metadata + checksum + refreshed info under exclusive lock
  4. Downgrade to shared
  5. Emit post-write hooks
  6. Release shared lock after PUT completes

This also makes the direct PUT path more consistent with ChunkingV2Plugin.

Scope

Only changes ordering during the final publish/finalization phase. Does not alter the part-file strategy, cross-storage move behavior, stream write path, or general hook semantics.

Review focus

  • Lock ordering during the final publish step
  • Checksum persistence ordering relative to post hooks
  • Shared-lock execution during post_write
  • Assumptions around batching metadata through putFileInfo()

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards
Copy link
Copy Markdown
Member Author

/backport to stable33

@joshtrichards joshtrichards added this to the Nextcloud 34 milestone May 15, 2026
@joshtrichards joshtrichards marked this pull request as ready for review May 15, 2026 14:13
@joshtrichards joshtrichards requested a review from a team as a code owner May 15, 2026 14:13
@joshtrichards joshtrichards requested review from ArtificialOwl, CarlSchwan, icewind1991 and leftybournes and removed request for a team May 15, 2026 14:13
@joshtrichards joshtrichards added 3. to review Waiting for reviews feature: filesystem and removed 2. developing Work in progress labels May 15, 2026
…mproving consistency

Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards joshtrichards changed the title fix(dav): finalize upload bookkeeping before downgrading lock fix(dav): finalize upload bookkeeping before post-write hooks May 17, 2026
@joshtrichards joshtrichards changed the title fix(dav): finalize upload bookkeeping before post-write hooks fix(dav): finalize upload metadata before post-write hooks while preserving shared hook locks May 17, 2026
At least where likely to be needed. Also fixed object type.

Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards joshtrichards marked this pull request as draft May 18, 2026 11:53
@joshtrichards joshtrichards added 2. developing Work in progress and removed 3. to review Waiting for reviews labels May 18, 2026
- `checksum` is already optional/derived metadata in practice
- callers already treat `null`l / `''` as "no checksum"

Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards joshtrichards changed the title fix(dav): finalize upload metadata before post-write hooks while preserving shared hook locks fix(dav): finalize upload metadata before post-write hooks May 18, 2026
@miaulalala
Copy link
Copy Markdown
Contributor

Ready for review? I'd love to have stable CI again 🙏

Copy link
Copy Markdown
Contributor

@miaulalala miaulalala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix — the core insight (keep the exclusive lock until all metadata is persisted) is correct, and the consolidation has a pleasant side-effect: checksum is now written in the same putFileInfo() call that writes mtime/ctime, so refreshInfo() finally reads the correct checksum. Previously setChecksum() ran after refreshInfo(), meaning the in-memory $this->fileView had a stale/empty checksum until the next request.

A few small observations:

getChecksum() !== null is now unreachable (nit)
FileInfo::getChecksum() now returns $this->data['checksum'] ?? '' (string, never null), so the null-guard in File.php can't fire. Worth removing or simplifying to !== ''.

touch() now runs under the exclusive lock
Previously the mtime/ctime writes happened after the lock was downgraded to shared. They now happen before, so they're serialised under the exclusive lock. That's strictly safer, just worth noting in case someone audits locking behaviour later.

$ctimeHeader !== '' is an improvement
The old truthy check would have rejected '0' as a valid ctime header. The explicit !== '' is more correct.

Test fixes look good
The TYPE_FOLDERTYPE_FILE change and the unrelated testPutLargeFile fix are pre-existing bugs, good to clean up here.

Missing concurrency test
Totally understood that a reliable test for the race is hard to write. A comment in the code pointing to the issue number would help future readers understand why the lock is held longer.

@joshtrichards joshtrichards marked this pull request as ready for review May 19, 2026 14:26
@joshtrichards joshtrichards added 3. to review Waiting for reviews and removed 2. developing Work in progress labels May 19, 2026
@miaulalala miaulalala self-requested a review May 20, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants